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Preface 


Documeniation standards and _ procedures for 
contracting out survey research has been singularly 
neglected by speakers and writers in the information 
research field. This lack of available reference material 
has been acutely felt by members of the Federal 
Statistical Activities Secretariat (formerly Special 
Surveys Coordination Division) of Statistics Canada, in 
their role as technical consultants to survey sponsors 
in other Federal Government departments. 


This handbook was prepared specifically for the 
guidance of survey sponsors and managers in the 
Federal Government, many of whom may have had 


little previous experience in the contracting out 
process. However, requests from industry and bu- 
siness schools indicate that the handbook has amuch 
wider application than originally anticipated. 


Preparation of this handbook has been greatly 
facilitated by the active cooperation of many indivi- 
duals in the Federal Government who have provided 
official source documents and personal expertise, and 
in the survey research industry who have provided a 
wealth of relevant information concerning the industry 
viewpoint. 


1. Introduction 


A person who Is new to the purchasing of research (as 
well as the purchaser who does not know how to define 
exactly what he wants to buy) can be confused by both 
the procurement system and differences among 
Suppliers, each of whom may have a slightly different 
service to offer. Unless the purchaser is able to devote 
a substantial amount of time and effort to learning 
about the procurement process and the differences 
among suppliers, the end result of the research may 
differ considerably from that which was intended. 


The purchaser of survey research services is particu- 
larly vulnerable in this regard. There are a number of 
significant differences between contracting for deliv- 
ery of some physical product and for the carrying out of 
a survey research project. A physical product can be 
measured, weighed, tested, analyzed and otherwise 
checked for conformity to fixed specifications and 
product standards. As a result of these checks, the 
product can be certified as within specifications and 
therefore acceptable, or outside specifications and 
rejected. 


The performance of a research service cannot be 
evaluated in a similar manner. Research is a commodi- 
ty produced by personal knowledge, skill and, quite 
often, originality. Consequently, the purchase of 
research services is literally the purchase of the 
knowledge, aptitudes and acquired skills of the 
research firm. It can be equated very closely to the 
process of hiring a skilled employee. Certainly many of 
the same principles apply. This situation is tacitly 
recognized in both policy statements and regulations 
on contracting. 


This handbook outlines the regulations, policy and 
recommended practices for the purchasing of com- 
mercial or academic survey research services by 
federal government departments and agencies. It is 
intended for use as a reference by survey sponsors 
and procurement officers who are involved in the 
contracting of survey research projects. The material 
presented is intended to reflect the intent of relevant 
official policy directives and regulations, but it is not an 
official interpretation and does not attempt to cover 
exceptional cases. A bibliography is included for those 
who have need for more details. 


2. Policy Considerations 


The Treasury Board has, over the years, issued a 
number of policy statements, guidelines and directives 
covering contracts, all of which observe these two 


principles: 
(a) To secure the best value for the money spent; 


(b) Within the limits of practicability, that all qualified 
firms (or individuals) should have an equal opportunity 
to compete for government contracts. 


In the case of survey research, a third principle also 
applies: 


(c) That a rationale has been established for obtaining 
the services of a commercial supplier. 


(a) Securing the Best Value for the Money Spent 


This principle is best explained by a direct quotation 
from “Policy and Guidelines on Contracting in the 
Government of Canada”: 


“The attainment of best value is a difficult and complex 
problem when one considers the various types of 
procurement that occur. It is recognized that there are 
acquisitions in which the requirements and specifica- 
tions are Clear, the reliability record of likely suppliers is 
relatively uniform and discretionary judgment is at a 
minimum with price or cost being the primary 
consideration. However, there are many other types of 
procurement in which the element of judgment 
increases in importance. For such types it is unwise to 
focus simply on price or least initial cost. There are 
many situations in which, for example, the goods or 
services that are being offered by different suppliers 
are not identical. Assessments and trade-offs are then 
required between different performance characteris- 
tics, costs, dates of delivery, service, follow-on 
procurement and logistic support...” 


“It follows that the necessary analysis to achieve best 
value should not be confined to the actual procure- 
ment process alone but rather should be pursued from 
the early stages of program planning and appraisal of 
alternatives through to the later stages of requirements 
definition, evaluation of sources, the selection of a 
contractor, preparation, negotiation, execution and 
award of a contract, the contract administration phase 
and the post-contract evaluation process. In other 
words, much of the required assessment of the various 
factors should be performed prior to the initiation of the 
contractual process...” 


(b) Equal Opportunity to Compete for Government 
Contracts (within the limits of practicability) 


This principle can be interpreted in many different 
ways, from the completely literal approach of giving all 
known firms an opportunity to bid for all contracts for 
work within and even beyond their stated capabilities, 
to an extremely selective approach wherein only one 
firm is deemed to be able to understand a department's 
problems. 


Under most conditions, neither extreme is particularly 
justifiable. Preparation of bids represents a major 
overhead cost to firms in the research industry (more 
about this in Section 6, “Inviting Bids’), and evaluation 


of an excessive number of bids is costly in time. The 
“Government Contracts Regulations” state that: 


“A contracting authority shall invite tenders ... from 
suppliers on a list that is, in the opinion of the 
contracting authority, representative of the suppliers of 
the required goods or services”. 


(c) Rationale for Using Research Suppliers 


When requesting authorization to obtain the services of 
a commercial supplier of survey research services, the 
sponsor will normally be required to prepare some sort 
of request for departmental approval to spend the 
funds. It will be useful to be able to identify a specific 
rationale for the request, such as: 


1. Specialized knowledge and skills can be obtained 
to deal with complex problems which are outside the 
skills or experience of existing staff: 


2. Objectivity and afresh viewpoint are available from 
outsiders who are not personally involved in the 
outcome of the project: 


3. Research can be performed promptly and on 
schedule because the required work force is already 
trained and in place at strategic locations across the 
country; 


4. Special interviewing facilities are required for 
conducting effective group or in-depth interviews; 


5. Manpower and cost savings can be achieved by 
reducing or eliminating the need for establishing and 
maintaining a full-time research staff in the sponsor’s 
department: 


6. Anonymity of the sponsor's organization may be 
desirable in some types of surveys, such as those 
involving attitudes and opinions. 


Once a logical and defensible rationale has been 
established, and provided that sufficient funds are 
available, the sponsor is ready to request departmental 
approval to carry out the project by the most effective 
means available. This may involve using either 
government facilities and staff or a commercial 
research firm. 


Contracting Authority of Departments 


Most departments and agencies may invite suppliers 
to provide proposals and negotiate a research contract 
directly with the supplier selected, without specific 
approval of Treasury Board, so long as the amount of 
the contract does not exceed the limit specified in the 
Government Contracts Regulations (see Bibliogra- 
phy). 


3. Preparing Survey Specifications 


Survey specifications provide a description of the work 
to be done, the amount of work, when it is to be done 
and, if relevant, how it is to be done. The supplier must 
not be expected to make his own assumptions 
concerning any significant factor because each 
survey research project is nearly always different from 
all others. 


In order to provide this range of details, the survey 
sponsor must carefully consider each step involved 
and evaluate the possible alternative approaches and 
methods which could apply to each step. To learn later 
that some step was done “the hard way” when an 
easier alternative could have been used is discon- 
certing and represents a needless expense. 


Specifications Should Include: 


(a) Objectives of the survey; 

(bo) A description of the needs of the sponsor (the 
sponsor’s interpretation of the objectives into specific 
requirements): 

(c) A description of the end use of the results; 

(d) Some indicators of the scope of the survey, and/or 
the available budget: 

(e) Nature and scope of analysis and reports required; 
(f) Materials and services (if any) to be provided by the 
sponsor; 

(g) Relationship to other surveys (if any) Concurrent, 
past or planned; 

(h) Any prior decisions concerning approach or 
methodology or questionnaire content; 

(i) Sponsor’s operational requirements; 

(j) Required qualifications of supplier. 


Not all items in this list will necessarily apply in every 
case, but any exclusions from the specifications 
should be the result of a conscious decision. 


(a) Defining Survey Objectives 


Objectives of the survey tend to be expressed in very 
broad and often oversimplified terms when they are 
first conceived. As planning progresses, it often 
becomes apparent that the survey objectives are 
so all-encompassing that meeting them fully would not 
be feasible. It then becomes necessary to convert the 
original vague terms into more Clearly-defined require- 
ments which can be used as the basis for taking 
specific action in terms of designing the survey. 
Satisfactory objectives for a survey can generally be 
derived from the answers to three simple questions: 
— ‘What are the circumstances that gave rise to the 
need for information?’ 
— ‘What is it that you really want to find out?’ 
— ‘How are you going to use the information ob- 
tained?’ 


This process of clarifying objectives can usually be 
done by breaking down the original problem (or 
purpose) into separate and meaningful parts, making 
each of these parts as clear as possible, and indicating 
possible solutions or hypotheses for each part. These 
steps may have to be repeated a number of times until 
the fundamental nature of the problem is understood 
and its components defined, before it becomes 
possible to place the problem within an operational 
framework of survey objectives. 


In this manner, problems are reduced to specific 
elements, relationships among these elements are 
considered in the light of practical experience and past 
knowledge, and hypotheses are formulated on what 
the solutions might be. After this is done, the results will 
usually be discussed with knowledgeable people in the 
field, examined in the light of other studies, and then 
revised further until a fully meaningful set of objectives 
is obtained. Exploratory interviews and other field 
research may also be undertaken, even at this early 
stage. 


These early pre-tests may indicate that some of the 
data are simply not available or that time limitations do 
not permit a survey of the desired scope, requiring the 
survey obectives to be revised so as to make them 
attainable. 


In cases where the problem appears to be quite 
complex and the sponsor does not have time or staff to 
define the objectives, the sponsor may contract out 
this essential phase of preparation to a consultant. This 
consultant may be a specialist in any of a wide variety 
of disciplines, depending on the subject area. The 
consultant does not carry out any portion of the survey, 
but only collects and organizes facts concerning the 
problem, and then assists the sponsor in preparing a 
plan for taking corrective action, including preparation 
of objectives and specifications for a survey. 


(b) The Sponsor's Needs 


The sponsor must convert the objectives into specific 
requirements for information which can then be 
considered to be and treated as discrete work 
packages. This item can be important in cases where 
the objectives could be interpreted in various ways by 
outside suppliers or where the sponsor has decided to 
split the project intoanumber of components, such as 
successive surveys, or to use completely different 
approaches or separate surveys to deal with different 
aspects of a complex problem. 


(c) Establishing End Use 


This item will enable the supplier to tailor* the overall 
approach, the actual questions, the processing and 
analysis, and the format of the final report in a way 
which will be most useful to the sponsor. 


ae 


Establishing a detailed profile of the end use of the 
results can also be extremely useful to the sponsor 
when writing survey specifications. To some sponsors, 
this approach may appear to be unduly restrictive, but 
it does tend to keep the specifications more realistic 
than they might otherwise become. For instance, will 
the results be used as input to one or more of the 
following: 


— Future policy decisions? 

— Facilities or operational planning? 

— Contingency planning? 

— Formation of expansion of a data base? 

— Planning of more detailed surveys to be conducted 
subsequently? 

— Public relations planning? 

— General knowledge about a situation? 

— Some combination of the above? 


Conversely, it may also be useful at this time to 
determine for which possible end uses the results will 
not be used. 


Once the end use has been clearly stated, the benefits 
will continue to grow. For example, selection of survey 
topics and questionnaire design will be simplified. In 
addition, the groundwork for analysis will have already 
been established. 


(d) Indicators of Scope and/or Budget 


Without Knowing the background situation or the 
relative importance of survey results to a sponsor, the 
Supplier may be unable to gauge the scope and 
amount of work or the accuracy of the results expected 
by the sponsor. In addition to a description of the 
sponsor's needs and a statement of the end use of the 
results, some specific indicators of the scope and size 
of the project should be given, such as: 


— Nature and complexity of the underlying problem; 

— Definition of the population to be studied; 

— Number of interviews or mail questionnaires to be 
completed, if known at this time; 

— Geographic area to be covered; 

— Desired level of accuracy of results, 

— Some indication of budget limitations, particularly in 
the case of creative proposals where the scope, 
overall approach and/or survey methodology have 
not yet been clarified. 


* Tailoring a survey does not mean biasing the results toward the 
sponsor's expectations. It means designing approaches and 
questions which best meet the information requirements of the 
sponsor. 


(e) Analysis and Reports 


There is no established standard for analysis and the 
sponsor must identify what is needed, usually based on 
the analytic capabilities available within the depart- 
ment. The sponsor shouid involve departmental 
analysis and computer staffs in developing this part of 
the specifications and, later, in communicating with the 
supplier selected concerning technical requirements. 


Reports covering survey results can have any format 
and contain any specified level of detail. Acomprehen- 
sive report may be requested, accompanied by all 
available supporting material. Alternatively the spon- 
sor may request only a condensed summary report 
supported by significant graphs or tables. Some 
sponsors request punched cards or tapes and/or 
printouts. In allcases, however, it is recommended that 
the details of survey methodology (e.g., method of 
sample selection, method of data collection, call 
backs, quality control, etc.) be requested, because 
awareness of the particular methodology used can 
greatly affect the analysis and interpretation of results. 


Reports may also contain recommendations for action 
to be taken by the sponsor, If requested. 


(f) Materials Provided 


In some situations, there will be voluminous back- 
ground material. The sponsor may have a suitable list 
of the survey population or of the proposed sample, or 
a draft questionnaire, available for use by the supplier. 
The availability of this material should be stated 
because it affects the amount of work to be done by the 
supplier. It may also affect the timing of the survey. 


(g) Relationship to Other Surveys 


A significant number of surveys are only part of a larger 

research project, such as cases where the current 

Survey is: 

— Exploratory and is to be followed up by more 
detailed surveys subsequently; 

— One of several surveys, each of which is dedicated 
to some particular aspect of a complex problem; 

— A detailed survey for which much of the input (e.g., 
concepts, assumptions, etc.) was obtained by 
means of earlier Surveys: 

— Required to be consistent with other surveys to 
ensure that results are comparable over time or with 
surveys in other areas. 


Unless bidders are informed of this situation, the 
specifications may appear to be incomplete or based 
on unwarranted assumptions, and the bidders may be 
confused or misled as to the precise scope of the work 
to be done. 


(h) Prior Decisions 


The decision to conduct, and then to contract out, a 
survey (or any part of it) may occur at any time during a 
complex review of a program. Consequently, much of 
the context and many of the parameters within which 
the survey must be conducted may depend on 
decisions which were taken without the survey in mind. 
Similarly, considerable work leading toward a survey 
may have been done before the decision was taken to 
contract out the actual survey. If any of these prior 
activities have established conditions or caused 
decisions to be made which affect the planning or 
conduct of the survey in any way, these should be 
described in the specifications. 


(i) Sponsor’s Operational Requirements 


Operational requirements consist of items such as: 

— Approval procedure (if any) for survey and ques- 
tionnaire design; 

— Progress reporting intervals; 

— Time restrictions, deadlines and completion date; 

— Access by the sponsor to completed question- 
naires; 

— Form of final report, scope, level of detail, personal 
presentation and interpretation of the findings to 
departmental staff, etc.; 

— Any other requirements peculiar to the study. 


Many of these requirements will affect the price quoted 
by bidders. For example, obtaining design approvals 
may involve preparation of successive drafts, repeated 
trips and numerous telephone calls. Personal presen- 
tation of reports and interpretation of findings by the 
Supplier may also represent significant preparation 
time and other related costs. 


An unrealistic Completion date deserves additional 
comment. The competitive bidding process almost 
always requires a number of weeks. The bidder who is 
then given the contract requires time to do an 
adequate job of background or exploratory research, 
planning, sample design, questionnaire preparation, 
pre-testing and assignment of survey staff. Unless the 
sponsor allows sufficient lead time, the supplier will be 
forced to cut corners and the quality of findings is likely 
to suffer. 


(j) Required Qualifications of Supplies 


In the process of defining the problem and preparing 
objectives, the sponsor will usually become aware of 
any particular specialized skills or facilities which the 
supplier must be able to provide. This requirement may 
go well beyond the normal skills or facilities provided 
by the majority of suppliers. The sponsor should outline 
such requirements in the specifications and bring them 
to the attention of procurement officers and members 
of the evaluation team. 


4. Confidentiality 


Confidentiality of Background 
Information Covering a Survey 


To obtain the maximum usefulness from the competi- 
tive bidding process in the form of “creative” 
proposals, a sponsor should not unduly censor 
background information or survey specifications to 
“launder out” sensitive, confidential or unfavourable 
details if these are relevant to the study. 


Survey research suppliers are generally highly ethical 
and are extremely careful to maintain confidentiality of 
information given to them, regardless of whether or not 
they are awarded a contract. Most of their business 
comes from a highly competitive private sector where 
unauthorized release of confidential information is 
definitely not acceptable. 


Suppliers of research services cannot prepare an 
intelligent proposal unless they have been given 
adequate background information and specifications 
of the work to be done. Without this information, a 
proposal tends to be either the bidder’s calculated 
guess of what the sponsor may want or merely a re- 
statement of the skills available and a description of 
other assignments which were carried out for other 
clients by the bidder. It cannot represent creative 
interpretations of how the bidder could best meet the 
sponsor's requirements. 


Those suppliers who have greatest experience with 
government activities will be at an advantage when 
specifications are vague and the sponsor refuses to 
provide sufficient information. In these cases, a 
supplier may either refuse outright to prepare a 
proposal under these limitations or may ‘dress up’ a 
relatively standard work package (which very often 
incorporates that supplier's favourite techniques) and 
submit a proposal which is based largely on intuition 
regarding the sponsor's probable requirements. 


Confidentiality of Bids Received 


Strict confidentiality should be given to the ideas and 
proposed techniques contained in bids which have not 
been paid for because, technically, these remain the 
property of their respective firms. 


In cases where a nominal payment has been made for 
bids, the sponsor is in a good bargaining position to 
negotiate for permission to use the ideas contained in 
the bids in any way that he/she wishes. 


9. Locating and Classifying 
Research Firms 


Survey research firms range in size from one-person 
firms to those employing over 100 full-time office 
personnel plus a national network of field interviewers. 
Some firms specialize in a “think tank” approach and 
have little or no field staff, others provide a wide variety 
of services, and still others consist almost entirely of 
field data collectors. 


Some firms have developed research expertise 

and/or facilities of particular types, such as: 

— In-depth interviewing; 

— Face-to-face interviewing; 

— Mail questionnaires: 

— Regular “omnibus” surveys which collect data fora 
number of clients from a pre-selected sample of 
known size taken from the general population; 

— Centralized telephone interviewing in a number of 
the larger Canadian cities: 

— Structured mail panels; 

— Shopping mall interview locations. 


Still other firms specialize in processing services such 
as data capture and tabulation, or are active only in 
management consulting. 


Locating Research Firms 


Repetitive purchasers of research generally maintain a 
list of those firms whose areas of activity are suited to 
the needs of the department. One-time or occasional 
purchasers who do not have such a list will probably 
wish to make inquiries concerning the qualifications 
and capabilities of each firm before sending the firm an 
invitation to bid on a contract. As a start, names and 
addresses of research firms can be obtained from: 
— Survey sponsors in other departments or specialist 
officers in the relevant discipline; 
— Directories (See Appendix “A’”’): 
— Personal search, professional associations, etc. 


Classifying Research Firms (i.e., Preselection) 


Before sending an invitation to bid to a firm with which 
he/she has not had previous contact, a sponsor may 
send a letter of inquiry to the firm in order to obtain a 
profile of the firm as a potential research supplier. In 
cases of large contracts, the sponsor (or one of his/her 
Staff) may also visit the firm to assess its capabilities 
and facilities. These and other inquiries will enable the 
sponsor to classify each firm according to those 
factors which he/she considers relevant. Factors 
generally considered are: 


— Subject-matter experience; 
— Technical competence; 
— Area(s) of specialization; 
— Educational qualifications of staff; 
— Reputation among government sponsors for quality 
of work, originality, prompness and relative cost; 
— Ethical standards; 
— Size of firm - 
— Geographic coverage, 
— Variety of services available within the firm, 
— Range of specialists on staff, 
— Adequacy of staff resources for emergencies; 
— Location (particularly relevant to regional surveys); 
— Language capabilities. 


Classifying research firms according to their qualifica- 

tions and capabilities is a general practice which is 

based on a number of practical considerations. These 
include: 

— The sponsor is reducing the uncertainty involved by 
dealing only with firms whose qualifications and 
Capabilities have been verified; 

— Firms which do not have the required qualifications 
and capabilities are not expected to incur the 
expense of preparing bids; 

— Fewer bids received mean less administrative cost 
to the sponsor through the reduction in processing 
and evaluation time required. 


A problem remains, however, where there are a large 
number of qualified suppliers but the survey to be 
contracted is quite small and therefore does not justify 
inviting a large number of bids. Sponsors who contract 
out a number of Surveys over a year solve this problem 
by dividing their list of qualified firms into parts and 
rotating through the list for each successive contract. 
This practice gives all qualified firms an equal 
opportunity to bid on at least one contract over a period 
of time. 


6. Inviting Bids 


Competitive bids should, in general, be invited by a 

department interested in awarding a contract. The 

reasons are: 

— To obtain the best value for the expenditure; 

— To provide an equal opportunity for qualified firms 
(and individuals) to compete for government 
contracts. 


Survey specifications should be in writing, without 
exception. Providing details verbally tends to lead to 
misunderstandings and even to disputes over what 
was or was not Said. 


Contents of An Invitation to Bid 


Before an invitation to bid is sent out, it should be 
checked over for completeness, with particular 


attention to: 

— Clarity of objectives and some indication of the 
priorities among the various objectives; 

— Adequacy of background information: 

— Adequacy of the statement of work required: 

— Available budget or some other indication of size 
and scope of project: 

— Completion date: 

— Restrictions and limitations (if any) (e.g., weather or 
other seasonal problems, required timing of phases 
of work, conditions for interviewing special popula- 
tions, etc.); 

— Any other requirements peculiar to the project: 

— Whether inclusion in an omnibus survey is accepta- 
ble; 

— Confidentiality of completed questionnaires: 

— Nature of reports to be provided; 

— Schedule for invoicing; 

— Name, telephone number and address of person to 
whom inquiries should be directed and to whom 
final report should be sent; 

— Date by which bids must be received: 

— Criteria which will be used to evaluate bids received 
(See Section 7, “Evaluating Bids). An aspect of 
these criteria which is important to bidders is 
whether the overriding criterion will be quality or 
price. 


Informing Bidders of Competitors 


A growing practice in Government is to incorporate 

into the invitation to bid: 

— (At a minimum) a statement giving the number of 
firms which are being invited to bid on this contract; 

— (Preferably) the names and addresses of the firms 
being invited. 


This practice enables prospective bidders to assess 
the capabilities of competing firms and encourages 
those firms with special capabilities in a particular area 
of study to devote considerable effort to preparation of 
their respective bids. 


Another benefit is that it indicates consideration for 
firms in the industry when a purchaser of research is 
required by outside pressures to send invitations to bid 
to a large number of firms. Each firm involved can 
judge its competitive position before going to the 
expense of preparing a bid. 


Selection of Firms to be Invited to Bid 


Reasonable judgment should be exercised when 
selecting the firms which are to be invited to bid and 
when deciding how many firms should be approached. 
Invitations to bid should generally be sent only to firms 
which are known to be interested in those types of 
survey research activity which are broadly similar to 
the project at hand. The number of firms to be 


approached is at the discretion of the sponsor or the 
procurement officer (or both) and may range from one 
or two up to about eight or perhaps ten. 


For smaller contracts (i.e., those under $5,000), the 
cost of preparing detailed bids may not be economical- 
ly attractive, and the sponsor or procurement officer 
may first wish to seek an expression of interest from 
one or two firms of known capability to ensure receipt 
of at least one serious bid. For larger contracts, at least 
three firms generally should be invited to bid. For 
contracts ranging beyond $100,000, the number of 
firms to be approached is normally increased to five or 
more but, in general, it is desirable to keep the number 
to ten or fewer. Beyond this range, firms may decide 
not to bid because of the economic cost of preparing a 
bid in the face of the increased competition. Another 
factor is the limited number of firms in the country 
which have the required resources to carry out a 
contract of this size without involving subcontractors. 


Sponsors responsible for awarding large contracts are 
advised to obtain a copy of the Government Contracts 
Regulations and/or to consult with the Contract 
Review Division of Treasury Board. 


Incidentally, advertising in newspapers is practically 
never done to locate prospective bidders for survey 
research contracts. 


Non-Competitive Bids 


Although competitive bids should be obtained under 

normal circumstances, the Government Contracts 

Regulations do permit exceptions to this rule. Justifica- 

tion for not obtaining more than one bid may exist in 

situations such as these: 

— The project is highly confidential; 

— Extreme urgency exists and the project cannot be 
subjected to the delays involved in obtaining 
competitive bids; 

— The project is an extension or replication of an 
earlier study carried out by a commercial firm and 
involved considerable preparation, testing and 
development of specialized procedures or of a 
particular collection vehicle or technique. 


Where it is decided to obtain only one bid, the following 
steps are recommended: 


1. The sponsor should identify the reason(s) for the 
strict confidentiality, urgency, etc., in writing, and 
place this statement on the file. 


2. The sponsor should use some form of pre- 
screening/prequalification process to select a 
supplier, and place on record the names of the 
suppliers who were considered and the reasons for 
selection of the preferred supplier. 


3. Where $15,000 or more is likely to be involved, the 
sponsor should consult the Contract Review Division 
of Treasury Board before negotiating a contract. 


4. The sponsor should request a written proposal 
from the selected supplier, outlining the circums- 
tances and reserving the right to reject all or part of 
the proposal if it is unsatisfactory for any reason, 
including cost. 


Prequalification of Bidders 


Prequalification is the process whereby a sponsor may 
invite firms to state their Competence, capability 
and/or unique skills for a specific project in response 
to a structured set of questions which have been 
developed from the specifications of the project. Bids 
are then invited only from those firms which appear to 
be qualified. The purpose of prequalification is to 
eliminate the probability of a firm being disqualified 
after having incurred a considerable expense in 
bidding on a large and complex project. 


Preselection and prequalification are similar, with the 

differences being: 

— Preselection consists of determining suitability of 
potential suppliers for projects of a broad general 
type, and is carried out prior to a specific need 
occurring; 

— Prequalification consists of determining the capabi- 
lity of specific firms to handle a specific project, and 
is Carried out immediately prior to inviting bids. 


The invitation to bid is not a suitable vehicle for use as a 
preselection/prequalification questionnaire because 
of other factors which may be involved, such as: 

— Afirmmay assume that an indication of interest (i.e., 
a bid) is necessary to remain on the list of firms, 
although the current project may be somewhat 
beyond the firm’s skill or experience, or the firm may 
Currently have all the work it can handle; 

— Bidding for a particular project does not indicate the 
firm’s Capability in general or its capability to do 
specialized research in other areas. 


Estimating Cost of a Project 


Sometimes it is extremely difficult for a sponsor to 
estimate the probable cost of a project or a sponsor 
may have had no previous experience in the contrac- 
ting of survey research. In such cases, the sponsor 
should attempt to obtain at least a rough estimate, so 
that the approximate cost is known prior to sending out 
invitations for bids on the contract. This eliminates 
situations where the only available course of action is 
for the sponsor to cancel the project because of 
unexpectedly high bids, but only after a number of firms 
have gone to the expense of preparing detailed bids. 


Sponsors can obtain an estimate of the probable cost 
of a project from either of the departments which have 
prepared directories of commercial research suppliers 
(See Appendix A”), or from almost any of the research 
firms directly. There is no charge or obligation for such 
an estimate. If an estimate is obtained from a 
commercial research firm, this should not be treated as 
a formal bid. Courtesy requires that this firm be given 
an opportunity to bid on the actual contract when 
invitations are sent out. 


Situations may occur where project costs cannot be 
estimated because a plan of action has not yet been 
decided upon. In these cases the sponsor may decide 
to invite creative proposals from research firms and to 
pay for these proposals. 


Payment for Proposais 


In situations where the best means of dealing with a 
complex problem is not evident, a sponsor may wish to 
obtain “creative” proposals from anumber of research 
firms. In order to foster serious consideration of the 
invitation to bid by these firms, the sponsor may 
arrange to offer payment for proposals. Although not 
often done, paying for creative proposals is a legitimate 
practice. The amount of payment offered is at the 
discretion of the sponsor, ranging upwards from 
perhaps $1,000, depending on the complexity of the 
project. 


Although the payment offered for proposals should 
never represent the full cost incurred by bidders, it 
does encourage them to generate truly creative 
proposals. Furthermore, by offering payments for 
proposals, the sponsor is in a favourable position to 
negotiate with bidders for permission to use the ideas 
contained in their creative proposals in any way. 


The basic procedure for arranging for payment is: 
(a) The sponsor will obtain prior approval by the senior 
departmental financial officer, for making payment; 
(b) Invitations will state that bidders may invoice the 
department for a specified amount; 

(c) The sponsor will verify each proposal as bona fide 
before sending the invoice to the departmental 
financial unit for payment. 


Revealing Budget Limitations 


Many experienced purchasers of survey research in 
government routinely include an indication of the 
amount of funds available for the contract, as part of 
every invitation to bid. Unlike a physical product, 
research projects can be difficult to define with 
sufficient precision to ensure a reasonably consistent 
interpretation by all bidders, particularly where creative 
proposals are invited. Where one or more bids are 
substantially lower than most (in the absence of a 
stated budget), it is generally because these firms 


underestimated the scope of the project, the problems 
involved, the level of quality required, etc. Of course, 
each bidder will likely provide a uniquely different 
approach to any project, and it is the purpose of the bid 
evaluation process to select the one which bestfits the 
sponsor's requirements. 


Alternatives Within Budget Limitations 


lf the estimate is greater than the available budget, a 

sponsor can restructure the project so that: 

— it can be carried out over two or more years, or 

— only essential components are contracted out to 
commercial research firms (where this is feasible), 
with the remainder of the work to be done by 
departmental! staff, or 

— a priority can be assigned to each of its various 
components, and this priority used to select 
components to be contracted out within the 
available budget. 


Another approach is for the sponsor to outline what 
he/she would like to do, together with a clear indication 
of the amount of money available for this project, and to 
ask bidders for creative proposals. Bidders may devise 
ingenious alternative approaches which were not 
contemplated by the sponsor. 


Cost Incurred by Bidders in Preparing Research 
Proposals 


Preparation of a proposal! in response to an invitation 
represents a significant risk expense to survey 
research firms, often representing 5% or more of the 
cost of the total project. This means that if five firms 
each spend 5% of the value of a prospective contract 
in preparing their respective proposals, collectively 
they have already spent 25% of the value of the 
contract. Four out of these five proposals will not result 
in a contract. On this basis, each firm must spend $250 
to attract each $1,000 of gross income. 


How is this 5% built up? Usually the person who 
prepares a proposal is a senior researcher. This 
person’s salary, plus secretarial and other support 
staff, plus long distance telephone calls (to check 
details with the sponsor or ensure the availability of 
field staff at the desired time) plus travel usually 
represents $300 or more per day. A moderately 
complex proposal could easily require a week or more 
to prepare, representing a cost of $1,500 or more. 


Fishing Expeditions 


It is not ethical to solicit bids in any of these situations; 

— Where a department does not have the necessary 
funds available, but proceeds on the rationalization 
that “it would be nice to know how much the project 
might cost and, anyway, proposals do not cost us 
anything”. 


— Where a department invites a number of research 
firms to submit proposals primarily to obtain useful 
opinions on how a project might best be underta- 
ken, but does not have a firm intention of awarding a 
contract for the project. 

— Where a department invites bids and awards a 
contract, then takes the best (and probably most 
costly) proposal received and uses it as a work plan 
for the successful bidder who was granted the 
contract on price alone. 


Queries 


Queries from bidders during the bidding period should 
be answered in writing with a copy to the sponsor’s 
contract file. Telephone answers to inquiries, as well as 
any information provided in personal meetings, should 
always be confirmed in writing by the sponsor. 


In cases where an inquiry is received concerning a 
critical item, this may indicate that the survey 
specifications were either incomplete or too vague. 
The sponsor should consider seriously the merits of 
sending the query and the answer to all prospective 
bidders, to insure equality of treatment. 


7. Evaluating Bids 


The sponsor should prepare bid evaluation criteria 
before seeing any of the bids. This ensures that the 
criteria reflect only the sponsor’s operational plan and 
priorities, and are not influenced by those approaches 
presented by bidders. 


Evaluation criteria should be unique to the current 
project. In one project, originality and ingenuity of the 
bid may represent the principal requirement, while in 
another the availability of a country-wide trained data 
collection staff may rank highest. Alternatively price 
may be the most significant factor. 


The factors listed in the “Suggested Evaluation 
Criteria” (below) are intended for use by the survey 
sponsor as a general check-list in the preparation of 
evaluation criteria for the current project. After listing 
those factors which apply, the sponsor will generally 
wish to amplify certain ones or break them down into 
their component elements. Sponsors may also wish to 
add other factors. 


Care should be exercised to keep each of the factors 
and the component elements as separate and distinct 
as possible to facilitate rating each without reference 
to other factors. 


When the listing of criteria has been done, the sponsor 
should consider which factors are most critical for 
effective implementation of the project, and assign 
rating values accordingly, from the most to the least 
critical. Certain factors may also justify having a 
minimum rating, below which the bid would become 


rejected outright. 


Sponsors may obtain assistance in evaluating bids 
from either of the departments listed in Appendix “A”. 


Suggested Evaluation Criteria 


— Does the bidder show understanding of the project 
by dealing with the basic assignment as spelled out 
in the objectives and in the sponsor's identification 
of his/her immediate need for information? Some- 
times, bidders in reputable firms get carried away by 
their enthusiasm and appear to be looking for a 
problem to which to apply their favourite techni- 
ques, instead of selecting the appropriate techni- 
que to fit the problem. 

— To what extent is the sponsor’s unique problem 
analyzed in the bidder’s proposal? Does it show 
evidence of originality and creativity, if these are 
called for? 

—Is the general approach well chosen? Is the 
proposed methodology appropriate for the type of 
project? 

— Is the work plan realistic and adequate? Are the 
various steps spelled out adequately to show the 
level of quality control which will be exercised? 

— Is the proposed work schedule reasonable? If the 
schedule appears unbalanced, it may be that the 
firm's resources are already heavily committed, and 
this project must await its turn for use of these 
resources. Is rigid adherence to the schedule 
important to the sponsor? 

— Are the project control techniques suitable and 
adequate for this type of project? Will the sponsor 
be informed of significant delays? 

— Does the bidder have the resources in place to 
Carry out the project? Some firms are equipped to 
Carry out only certain phases of a project and must 
sub-contract other phases. Is the sponsor prepared 
to accept this? 

— Does the bidder give indication of having reserve 
resources available to cope with delays or unex- 
pected problems? 

— Does the bidder give evidence of financial stability 
and overall capability of meeting the contractual 
commitments? 

— Does the bid identify the persons who will actually 
work on the project, or does it list only partners, 
principals and senior analysts whose only involve- 
ment is to attract a contract and deliver the final 
report? (In the private sector, personal involvement 
of specified researchers is sometimes considered 
more important than the reputation of the firm.) 

— Are the qualifications and experience of the 
designated individuals spelled out? 

— Does the bidder agree to abide by the terms, 
conditions and timing specified in the invitation? 

— Lastly - Is the price competitive for the amount of 
work proposed? 


Recommended Bid Evaluation Procedure 


1. An evaluation team is selected, consisting of three 
or more persons including the sponsor. This number 
may be as high as five or six in the case of contracts of 
$100,000 or more. All team members should have had 
some involvement in the project and may participate in 
preparing the evaluation criteria. 


2. Each bid received is evaluated by individual team 
members against the previously prepared evaluation 
criteria and the survey specifications. Each rating is 
recorded on an evaluation sheet which may also 
contain very brief comments. 


3. After all bids have been evaluated, the team meets 
and members compare their respective ratings. 


4. Individual factor ratings which differ significantly 
among members are discussed and reconciled. 
Simple averaging of extreme ratings among team 
members should be avoided because it is not a 
defensible procedure. 


5. Bids are ranked by their averaged total rating. 


6. The team may decide to contact any of the firms in 
order to clarify particular points. However, if negota- 
tions are held with any of the firms prior to the selection 
of the successful bidder, all of the firms which qualify 
should be given an equal opportunity to modify their 
bids (to adjust minor details, for example), so long as 
this does not affect the price quoted in the firm’s 
original bid. 


7. The successful bidder is selected. Deliberations of 
the selection team are confidential. However, the 
substantiation of the final selection should be in writing 
and must be suitable for disclosure to the public, 
particularly to unsuccessful bidders. It must also be 
available for post-audit. 


8. The sponsor (and his/her procurement officer) 
negotiates any remaining details with the successful 
bidder and prepares a contract. 


9. Considerable goodwill can be generated for the 

sponsor: 

(a) (at a minimum) by promptly informing all unsuc- 
cessful bidders of the name of the successful 
bidder; 

(0) (preferably) by also stating the principal reason for 
the selection of the successful bidder. 


What a Bid Should Contain 


A bid should be a complete response to the sponsor’s 

invitation to bid, and normally should not require further 

amplification or explanation. Consequently, a bid 

should, in general, contain the following points: 

— The bidder's interpretation of the sponsor's descrip- 
tion of the project and its objectives, particularly if 


the original specifications are vague or if any details 
were expanded in discussion between the sponsor 
and bidder: 

— A description of the bidder’s proposed approach, 
usually called a “creative proposal’ when the 
approach and techniques to be followed are not 
outlined in the specifications or where the bidder 
feels strongly that the sponsor's approach is not 
suitable for the project; 

— A description of the proposed work plan including 
survey size, outline of the type of questions to be 
used, methodology, quality controls, scheduling 
and completion date; 

— A description of any sub-contracting of part of the 
work to other firms or to individuals who are not 
regular employees of the firm: 

— A description of the type and amount of resources 
planned for each phase of the project, particularly in 
the case of a large project; 

— A statement concerning any materials or services 
to be provided by the sponsor and their effect on the 
specified completion date; 

— A statement concerning any formal briefing ses- 
sions or other meetings requested by the sponsor; 

— A description of the reports to be presented to the 
sponsor, both progress reports and the final report; 

— Acceptance, or otherwise, of terms and conditions: 

— Price, with a reasonable breakdown of the project 
by components, and the contribution of each 
component to the overall price; 

— Identification of the staff.members responsible for 
the project and their qualifications; 

— Related experience of the firm and of those staff 
members designated to work on the project; 

— Stability of the firm, adequacy of reserve resources 
to ensure implementation on schedule, etc. 


Problems of Direct Comparison of Unit Costs 


When analysing cost figures provided by bidders, it is 
sometimes unwise to make direct Comparisons 
between the per-hour or per-day costs of apparently 
identical services. For example, one firm may quote 
direct operating costs only, while another may include 
hiring, training and supervision in their rates. 


Selection of Other than Lowest Bid 


When other than the lowest bid is selected, this 
decision should in all cases be based on the 
requirements laid down in the original specifications. If 
the lowest bid fully meets the requirements and a 
higher bid also does so, plus providing a desirable 
additional feature not called for in the specifications, 
this is not justification for selecting the higher bid. 


Unusually Low Bids 


Where one or two bids are conspicuously lower than 
the majority, these should be treated cautiously 


because, in most cases, either the bidders do not 
appreciate the amount of work involved, or are 
planning to conduct a superficial project. Of course, 
there can be exceptional cases, but in general, major 
differences in cost estimates should be analyzed 
closely. 


Where bids vary widely in quoted price, this may be an 
indication of inadequate specifications, causing widely 
divergent interpretations to be made by bidders of the 
scope of the project. 


Unsolicited Bids 


Firms in the research industry who were not invited to 
bid may learn of the proposed contract and request a 
copy of the specifications, resulting in an unsolicited 
bid. If the conditions and requirements are met, an 
unsolicited bid should be considered along with those 
which were invited. 


Complaints 


Complaints may be made that a firm was not invited to 
bid on a particular contract or that irregularity occurred 
in the selection process. For the occasional purchaser 
of research, this can be disconcerting. Because of the 
possiblity of this happening, it is desirable to have well- 
documented records of the supplier selection and 
procurement process, such as: 


— Survey specifications which identify any unique 
research skills required, completion date required 
and any special conditions or requirements; 

— Use of a recognized procedure for preselection 
and/or prequalification of known suppliers; 

— An accurate list of those firms which were invited to 
bid; 

— A detailed record of evaluation of bids and of the 
reasons for selection of the successful bidder. 


8. Awarding the Contract 


After the supplier has been selected, the sponsor (or 
his/her procurement officer) will be required to 
prepare a formal written contract for the signature of 
authorized representatives of the department and the 
supplier. A meeting or series of meetings may also be 
held with the supplier to set up working arrangements, 
provide background briefings and clarify details. 


Content of Contract 


The contract should contain the following: 

— A statement of project objectives; 

— The agreed-upon work plan; . 

— Acompletion schedule, progress reporting require- 
ments, etc.; 

— |dentification of technical details such as sampling 
arrangements, operational restrictions, etc., 

— Specifications of the processing and analysis to be 


done by the supplier and of the final report to be 
prepared by the supplier: 

—A statement of materials and services to be 
provided by the sponsor: 

— A statement of the quoted cost: 

— A statement covering invoicing requirements: 

— Default and cancellation provisions, terms and 
conditions. 


When signed by an authorized representative of the 
department and the supplier, the contract is a legal 
document which cannot be changed, except by mutual 
agreement (other than outright cancellation by the 
department). 


Changes 
Contract 


Requested During Preparation of 


Occasions arise where a supplier requests significant 
Changes to or expansion of a project during the 
preparation of the contract, with a corresponding 
increase in cost to the sponsor. These requested 
amendments to the contract should be rejected, as the 
supplier is requesting that the contract be materially 
different from the specifications on which bids were 
invited. 


Similarly, a supplier can refuse to sign a contract if the 
sponsor includes significantly more work in the 
contract than was described in the invitation to bid. 


If the selected bidder identifies a major error in his/her 
bid before the signing of the contract, the bidder may 
submit a corrected bid. The sponsor will normally then 
re-open the bid evaluation process and negotiate a 
contract with the lowest qualified bidder. 


Amendments to Contracts 


Amendments can be made to contracts in cases 
where the supplier must carry out additional work 
beyond that specified in the contract, in order to meet 
the requirements of the contract. Such situations may 
arise where the sponsor is unable to provide the 
materials or services originally stated, or where 
unforeseen events occur. 


Amendments to contracts can be the subject of 
intense negotiation and are outside the scope of this 
handbook. 


General Comment on Preparing Contracts 


Preparation of the actual contract is a specialized 
operation and should be done in conjunction with an 
experienced procurement officer. Contracts follow a 
prescribed format and should contain certain specified 
terms and conditions, as well as provision for default 
and cancellation. 


9. Working with Research Suppliers 
Contact with Supplier 


The sponsor should set up a contact point for dealing 

with a research supplier immediately on the awarding 

of a contract. This contact may be either him/her self 

or a knowledgeable staff member. The duties will 

include: 

— Providing background and technical information 
requested or required by the supplier; 

— Arranging for materials and services to be supplied 
by the sponsor, as stated in the contract; 

— Arranging for immediate approval by the sponsor of 
survey plans, questionnaires, etc.; 

— Reviewing and responding to progress reports; 

— Responding immediately to unexpected problems 
which the supplier should not be expected to nandle 
or which may seriously affect the survey (e.g., 
misinterpretation of questions by respondents). 


Observation of Survey Activities 


The sponsor may arrange for one or a few departmen- 
tal staff members to observe survey activities. 
Observers are interested spectators only and must not 
act as quality inspectors or field supervisors. The 
Supplier is exclusively responsible for conducting the 
project. 


A sponsor may find observers useful for these reasons: 
1 - They increase feedback on progress of the project 
and reduce the need for formal progress reports; 
2- They observe field problems directly and may 
assist the supplier in overcoming these problems; 

3- They may learn facts outside of the original 
concepts of the problem; 

4 - They may learn useful survey techniques; 

5 - They can provide the sponsor with a detailed 
interpretation of project problems and of the survey 
results — thus eliminating surprises on receipt of 
the final report; 

6 - They can evaluate the supplier. 


A sponsor may find observers a liability for these 

reasons: 

— Their presence may annoy the supplier and cause 
tension; 

— They may become too involved in the project and 
interfere in its management or be constantly 
underfoot and impede implementation; 

—In some cases, their presence may introduce 
certain response biases. 


Notification to Departmental Personnel 


Where departmental field staffs may be involved in any 
way with a survey, either as respondents or as a result 
of inquiries from the general public, all members of 
these staffs should be notified in writing concerning the 
survey, its scope and purpose. Members of the general 
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public may, for instance, contact a departmental local 
office to check on the credentials of interviewers, and it 
is highly desirable that the local office staff can 
respond suitably. 


10. Evaluation of a 
Supplier's Performance 


On completion of a contract by a commercial survey 
research firm, the sponsor should prepare an evalua- 
tion of the firm’s performance. This evaluation may 
consist of informal notations in the case of a small 
contract, but should be more structured and compre- 
hensive for large contracts. The purposes of this 
evaluation are: 

— To provide a written record of value received for the 
expenditure made, for departmental use; 

— To record the quality of the firm’s performance on 
the sponsor's or departmental list of research 
suppliers, for future reference in connection with 
other contracts; 

— To comply with audit requirements of Treasury 
Board. 


The principal points which should be covered in an 

evaluation include: 

— A description of the work undertaken; 

— The quality of the work performed (measured 
against the original survey specifications); 

— The skills and general capabilities of the firm’s 
research and/or survey staff; 

— The quality of communication maintained with the 
sponsor during the project and in the interpretation 
of the survey findings: 

— The firm’s adherence to schedule during the 
project, and the timeliness of the final report; 

— An assessment of value received for the expendi- 
ture; 

— Comments regarding the possible selection of this 
firm for any future contract. 


Evaluation should be carried out promptly while the 
relevant facts are fresh in the sponsor's mind. 


11. Requirements for the 
Success of a Contract 


The success of a survey research project depends 
largely on the thoroughness of early planning of the 
project and initial testing of the concepts, followed by 
preparation of suitably complete survey specifications. 
These specifications provide the basis for all subse- 
quent phases of the project. 


Government Contracts Regulations, together with the 
various policy statements and guidelines issued by 
Treasury Board, provide the general framework within 
which a government sponsor must negotiate contracts 
with any outside suppliers. Except for certain basic 


requirements which are mandatory, they do permit a 
sponsor reasonable freedom to negotiate a suitable 
contract in order to achieve his/her objectives as 
effectively as possible within the available budget. 


The sponsor, in turn, is required to be aware of the 
regulatory requirements, to act in an organized and 
defensible manner, and to maintain a written record of 
all aspects of supplier selection and contract negotia- 
tion. 


Appendix “A” 


Directories of 
Commercial Research Suppliers 


The most readiiy available directories of commercial 

Suppliers of research are these: 

— Directory of Survey Organizations, prepared and 
maintained by the Special Surveys Coordination 
Division of Statistics Canada. Suppliers are classi- 
fied by general type. This Directory is available on 
request. 

— Establishments Listed in the Science Source 
Information System/DSS, prepared and maintained 
by the Science Procurement Branch of the 
Department of Supply and Services. Several 
different lists are maintained, sorted by the 
expressed special interest areas of the firms 
themselves. These lists are primarily for depart- 
mental use. 


Various industry associations exist but none repre- 
sents the entire industry. 


Appendix “B” 


Definitions of Selected Terms 


To assist in the interpretation of this handbook, 
definitions of a few selected terms are provided. 


Bid - an offer to carry out a defined project or 
assignment of work at a stated price (interchangea- 
ble with ‘tender’, “proposal” and “creative propo- 
sal”). In this handbook, “bid” is used where the 
context emphasizes the complete offer, including 
price. 


Creative proposal - a proposal wherein a supplier 
provides considerable originality in preparing a plan 
for dealing with the problem outlined in an invitation 
to bid. 


Invitation to Bid - a formal (written) letter or form 
which invites firms to bid on a proposed contract 
(sometimes called “request for proposal’ or 
“invitation to tender’). 


Letter of Intent - not used in government contrac- 
ting. 


Proposal - same as ‘bid’. In this handbook, “‘propo- 
sal” is used where the context emphasizes the 
Supplier's ideas and approaches contained in the 
proposed plan for dealing with the sponsor's 
problem. 


Sponsor - the person in a department or agency 
responsible for a project and who can or has 
obtained approval to spend the necessary funds. 


Supplier - a firm (or organization or individual) which 
is Capable of carrying out a contract, normally for a 
specific type of work or product. 


Tender - same as “bid”. “Tender” is not used in this 
handbook because the term infers a price quotation 
for a contract where the product can be clearly and 
rigidly defined. 
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